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imoDDcnoN 

The material reported in this presentation was developed as part of a 

systematic effort to develop techniqties to evaluate the effectiveness of a 

natflonalXy implemented early childhood program. • The Tucson Early Education 
♦ 

Model -has been supported for eight years in delivering an innovative educational 
approach to 19 school districts through funde . from the Follow Through Division, 
U* S. 0. E. Addiyion^l early support was given by the Office of' Child Develop- 
ment, which supported the implementation of TEEM lit three communit;ie8 ^t the 
Head Start level^*for three years. The longitudinal focus of Follow Through 
has permitted the development of a systematic evaluation program to assess 
especially the process dimenaions which have/ heretofore eluded formal 
measurement. / /"n^ 

The Tucson Early Education Model is a ptocess-oriented educational System 
with, f pur. are ad of program^objectives for children: Language Competence, 
Intellectual Base, Motivational Base, and Societal Arts and Skills (Arizona x 
Center for Educational Research and Development, 1974). These goals areas 
are met through an integrated classroom environment which emphasizes the 
orchestrated development of behaviors in children. Other process variables • 
focus on individualization of instruction, the provision of a classroom 
environment that provides gratification for children, and use of modeling 
procedurfe'fi^ to facilitate children* s learning. This emphasis on a process 
approach to learning, as well as the focus on the whole child, lead to jthe 
description of the TEEM as on open classroom program (Newsweek, May 5, 1971). 

The delivery system for TEEM relies on the "multiplier effect" ta make a 
minimum of training time and effort produce maximum impact on children in the 
participating Follow Through communities. The system of educational services 
.has three comppnent^ at the community level: (1) .classroom instructional 
staff, (2) parent liaison personnel', and (3) school psychologists. 
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Instructional, xPsycholog^cal Services, and P^irent Involvement fi^ld 
representatives fipm the A^fizona Center and their community counterparts 
(program assistants, pareint coordinators, and psychologists) play key roles 
in the delivery system. Thfey provide- the vehicle for information transmission 
among ays tern components and the means for implementing the multiplier effect. 



METHODOLOGICAL REVIEW 

In view of the need to develop new evaluation strategies, we decided on 
the development of situational tasks that would provide information about 
process goals in the open classroom. A sitjuational technique gathers informa- 
tion within an ongoing classroom setting, so thafr the information reflects 
the real-world behavior of children, and teachers. An additional feature is the 
introduction of environmental manipulation which puts performance demands on 
the classroom members. Such events as fire drills or visitors naturally occur 
In all classrooms. In this research technique, quasi-naturalT events are 



simulated and used as opportunities to collect observational data on the 
continuli^g behavior of classroom participants. 

Sittiational tasks were described by Grimmett (1970) as useful procedures 
/.to assess the effects' of experimental programs on the attainment of connativa 
and motive behaviors in childreh. She notesr the inadequacy of traditional 
paper-anti-pencil techniques fo^ gathering information in these crucial areas 

'of child development. A situational task (which she calls "situational test"; 

i 

1970) is defined as "a conditi|Dn requiring an actual, adaptive response, rather 
than a mere 'test' response . |. .£^dj. . . problem confrontation, the 
resolution of .which has some relevance for the 'fifal world'", (p. 12). These 
techniques are especially appjlropriate for assessing children's social perform- 
ante within the context where it was leame^d (i.e., in the classroom).. 
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This paper will describe the development of two different situational 
tasks .developed for evaluating the TEEM program. The extensive develop- 
mental effort has led us to an interest in disseminating these techniques 
to get wider feedback on their general usefulness to the field. The first 
technique is called Classroom' Attitude Observation Schedule (CAOS), 
developed to assess children's development of *an independent learning 
style. The second is the Children's Language Assessment-Situational Tadks 
(CLA-ST), designed to assess children's language development in a natural"^ 

setting. 
« 

The task in developing CAOS was to devise an evaluation technique to 

assess how effectively, indeed, children do develop as independent learners 

within classrooms implementing the TEEM. A review of available literature 

i 

indicated no extant studies with an adequate methodology to research 
"independent learning behavior" (Simon and Boyer, 19Z0). Thij^ led to a 
review of available observational research techniques with a view toward 
•develoi>lng a totally new technique, or modifying existing procedures to this 
end. At this point certain m;Lnimal a^fiteria were set to develop a novel 
evaluation technique. The following Criteria were set: (1) the technlflue 
should reflect actual behavior of children; (2) it shpuld be unobtrusive; 
(3) it should be valid with children from preschool through grade 3. 

Systematic observation of social interaction has only recently achieved 
respectability as a tool in educational research. , Initial ;work by White 
and Lippitt (1960) 'and Flanders (1966) .indicated th^ potential of observational 

methods to gatjier information on previously unres^arched vairiables of classroom 

• ♦ 

process. The Classroom Observation Instrument, developed by^Stallings (1975) 

gathers information about the nature of classroom interaction, the type of 
activities and group patterns, and the quality of the physical plant (S tailings, 
1972). Classrooms are observed over a three day period, for a total of 36 
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five-minute interaction sequences. A review of this procedure indicated - 
certain features that wduld be adaptable to the task of assessing Independent 
learning beHavlor in chllldrea. With the encouragement of COI developers, 
we built upon their initial instrument to meet our research goal* 

The Classroom. Attitude Observation Schedule was Sesigned to detect 
pattern shifts in selected process variables in a classroom during the 
absence of the teacher and other "controlling" adults. These variables 
^4re grouping patterns , tjrpes of classroom activities , and the occurrence of 
.^ ^^^^^^^^^^^^^ behaviors . In order to investigate these patterns, the observa- 
trdh.is divided into three phases. During the first twelve minutes (called 
Baseline phase) , the observer records the activities of adults and children 
on the CAOS schedule; during the second twelve minutes (called Teacher 
Absent phase) , the observer records the activities o£ children while the 
teacher figures are absent; during the third twelve minutes (called 
Reinattttition phase) , activities of both children and the returned teaching ' 
^ figures, are recorded. The total CAOS observation takes 36 minutes, and is ^ 
descri|rMvip Figure 1. , 

; ' . B • c 

''"i '-^M^W^^' 12 min. 12 mln, 

* Adultrff'-fresent Adults absent Adults reinstated 

(BASELMIE) ' (T ABSENT) (T' RETURN) 

X.X-JCXXX XXXXXX .* xxxxxx 

\s Fig. 1. Phases of CAOS. 

The observl^ipnal technique counts children and adults engaged in the 
various classroom activities on a time-sample feasis. All t3rpfes of activities 
presumed to take place in ^he classroom are \isted on the recording form 
(see Figure 2). Once every two mii|utes a clockwise visual sca^ is made of 
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the rooin by the observer^ The observer remains stationary throughout the 
36 minute. period, as the scan begins and ends ^t the same point for each 
scan. Numbers of children and aiiults observed during that scan ^re placed 
in the appropriate cell, thus retaining grouping patterns in the recording. 
If inappropriate behavior is observed during this scan, it^is also noted by 
its associated activity and in the appropriate two-minute scan period. Two 
-more scans are made during the two-minute period *to pick up incidents of 
Inappropriate behavior, once at the end of the first minute, and again at 
the end of. a minute and a half. , 



Insert Figure 2 about here 

Procedures and categories- frojn two previously developed instruments 
wer.e synthesized to produce this particular procedure. ^ Both of the parent 
procedures have been field tested and found reliable/ As a preamble to each 
interaction recording period, tile observer takes a "snapshot" which gives the 
. foundation for the CAOS technique. Activity definitions used in COI have 
been redefined to make them more program specific to the Tucson Early Education 
Modfel. 

The other instrument (Schedule for Incompatible Learning Behavior - SILB) 

(Grfmmett, Underwood and Brackney, 1970), was originally developed for a study C 

assessing the relationship of behavloi* settings to disruptive or inappropriate 

behavior ♦ The inappropriate behaviors selected fo/ coding in CAOS were: 

Hitting Yelling ^ 

Interfering Throid.ng 
Leaving room without permission 

These categories were developed and operationalized in consultation with 

. classroom teachers. Initial studies indicated that these categories could be 

reliably rated by trained obsei^ers. 
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PILOT STUDY 

The pilot study was carried out in a middle-sized commtpity in the Great 
Plains, which had a total of eight classroonjs operating in t^eir Head Start 

• ■ . 'J . 

program (six of which used the TEEM model, and two using locally- implemented 
curriculum) • 

The purpose of the CAOS pilot stud^ was to discover pattern shifts given 
the absence of classroom "controlling" adults. Such shifts could take many 
forms. The variables described below aire those that might demonstrate 
controls which are largely adult- centerea versus controls internalized by the 

children or. imposed by the physical ot bel^avlor setting. A n^eaningful pattern 

• I ' ' ' 1 ' 

would be one in which the variable Increases ,or decreases during teaclxer 

absence and returns to the level ekhibited during Phase A, wheln teacher returns 

in Pliase C. A stable pattern, then, would be one in which allUhree phases 

look much alike. * ^ 

The summary observation variables were: 

\ 

1. ■ Mean group size • ^ 

2. Mean number of children ""engaged in an activity 

3. Inappropriate behavior * 

4. ^ Mean number of groups 
^ 5. Mean number of activities 



Insert Figure 3 about ;here 



The most conspicuous difference appeared \kth the summary variable ' 
inappropriate bqhavlor. Analysis of this variali^ indicated that signifi- 
cant differences existed in both Independent variables, group and phase, 
as well as the interaction between these variables (group: p<.10; phase: 
p<.001; group, by phase:, p<.05). Figure 3 gives graphic demonstration of 
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the group-b^-phafee interaction. 
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TEEM (M = 6) 

Comparison (N = 2) 




A 

Baseline 



r 
B 

Teacher Absent 



Observation phase 



T 

Reinstitution 



Figure 3. Incidents of inappropriate learning 

behavior in TEEM and Comparison classrooms 
across observation phases.^, 
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When proportion of adult participation in each of tihe five activity 
types was compared to proportion of child participation in th^ s^e activity 
types during the same phases, rank drder correlation between adifft participa- 
tion ^d cKlld particiijation in the low^ raffed classroom was -.27 (Figure 4), 
^ In the high-crated c^lassrood t:he correlation between adult and chiXd participa- 
tion was ,80. . (See Figure 5.) 




CONaUSIONS i- CAOS 

The CAOS system is clearly sensitive to pattern shifts Iri some* aspects of^ 
chljji^behavioi^. The clearest pattern sh^ft appeared with levels of inappro- 
priate behav±o,r. Children in the two comparison classroonS displayed more 
inappropriate. behavior during the teacher absent phase than did the children 
in TEEM classrooms. The post hoc ^sts demonstrated no significant differences 
between level of inappropriate behavior in TEEM classrooms and Comparison 
classrooms during the teacher present phases. The differences between th^se 
two groups during teacher ajbsent phase was significant. 

With the TEEM classrooms, children in the classroom rated lowest *by the 
Head Start Director displayed more inappropriate behayior during teacher 
absence than did children In the high-rated classroom.^ It also held ttue ^liat 
there were significant differences between teacher" absent and toucher present 
phases in Comp^arison classrooms, while there were^no significant differences 
'between phases in TEEM Classrooms* These comparisons also existed ^ 
between the low-rated a^d the hig^-rated classroom. ' t 
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CHILDREN'S LANGUAGE ASSESS^ENT-SITUATIONAL TASKS 

The goal in developing CLA-ST was to devise a technique to assess child- 
ren's language development in a natural setting. Again, the instrument 
criteria were considered (see p, 3) in assuring a technique applicable over the 
early childhood period. Literature review indicated that a number of studies 
of children's language development had been completed, but they typically 
depended igpa children's writing. It was concluded that natural speech formed 
a much richer avenue for exploirlng language development than written products. 
Recent studies indicated that many seemingly non-verbal children really 
opened up when language was unobtrusively recorded on felie^layground (Conrad, 
Rentfrow, & Meredith, in press). Language development in the TEEM classroom 
is based upon the Language Experience Ap^proach-, in which the child's natural 
language is used as a base for evolving iaW formal standard English. 

The CLA-ST w.as developed to collect language saiiq)les, within a normally 

operating classroom The language is taken on a cassette tape recorder, which 

is placed at the foot of a small tabl^* At thi^ lable, in a committee setting, 

four children are engaged with a teacher in an activity similar to those they 

encounter dally. The CLA-ST Is broken into three segments: the first, with 

their own classroom teacher, ^utilizes a "mystery, bag" containing 27 common 

household objects (15 minutes) (see Figure 8); the second, with a new adult 

(a staff research assistant) , uses a set of eight picture cards that ha^e no set 

* / 

solution (15 minutes); in the third, the, children are left alone fox five minutes 
with the tape recorder running, while the adult excuses herself briefly. 

These three segments are intended to get different perspectives on children's 
language development. The first, with a familiar teacher, should resemble very 
closely their,.£yplcal use of language in the classroom. The second segment assesses 
whether tlielr language style is modified in. the presence of an unfamiliar 
adult. During Task II (strange adult) , the adult uses only a limited set of 
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artists brush 
snap beads (2) 
4 In. utility candle 
left hand dtten 
3 in. chain 
sea shell 
rubberband (h in.) 
red lead pencil 
flnlshdLng nail 



Fi22-Whl22 l?ottle cap 
red velvet ribbon (6 in.) 
orange balloon 
12-in. yellc^T^ler 
spool of green thread 
plexiglass (4: X 6 in.) 
saadpape-r (4 in* square) 
gireen button 
portion of egg carton 



2 in. cube yellow sponge 
roll of #20 wire 
marshmallow 

pink eraser 
4 din. nail 

3 pipecleaners 
paper clip 
metal brace 
green washer 



Figure 8, Contents ofbfystery Bag 
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,pron5)t8 to reduce spurious influences on the children's language. These pronjpts 
are given in Figure 9* The third segment seeks to simulate other studies of 
natural language, and the children are left alone with the cards from Task II 
and the tape recorder mnning. 

The tapes are returned to the Arizona Center for transcription and 
analysis. The information is first broken down into free flow fc^rmat which 
identifies teacher and child output. These transcripts are thfen submitt^ed 
to an analysis for basic language dimensions. 

A pilot study was conducted in Spring, 1973, and a total of 96 cLassr'pom 
units were saxapTed. Half of these were TEEM, and half were locally • selected 
Comparison classes^ The tapes included equal number of first, second, and * 
third level children, and sampled four diverse school districts using the 
TEEM program. Upon return, approximately 10% of the tapes were discarded as 
being unintelligible. ' 

Tbe^ following matrix of language dimensions was selectei for. analysis : 



LANGUAGE OUTPUT Average Nunier Words/Child 

Average Response Length 

LINGUISTIC DIVERSITY Type-Token Ratio 

Form^Function Ratio 

Present Tense Verbs /Total Verbs 

Vocabulary Diversity 

SYNTACTIC PERFORMANCE Number of Complete T-Units * - 

T-Units (Totals and Types A, B, C) 
Child-Initiated Questions 
Frequency of Role-Playing (Session II only) 

Figure 8» CLA-ST Analysis Variables 
Coding procedures used a consensus approach. Twenty per cent of the tapes were 
recoded and the resulting reliability for all language variables was 96%, with 
a range of 100% to 79%. ' 
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Tabi^l smmnarizes .the results from the pilot study. Due to 
loss of tapes, ANOVA procedures were not usable* Thus, sign tests 
were appliedXto these data. The results Indicate that TEEM child- 
ren vere espe^ally productive in terms of using more words over 

sessions, having a more diverse vocabulary^ and askings moxe questions. 
\ * 

Comparisons were especially strong in terms .of T-^Unlt Type A 

\ ' 

(functionally complete, structurally incomplete) . 

The results from the pilot study -indicated that the CLA-ST is 
a useful technique fo±^ sampling chlldrenVa language! Another study 
has been Initiated, .to A^rther develop the -technique and to develop 
validity information.. TMs study is ongoing t^iis year, and language , 
samples' have been collected in 20 classrooms in a rural Applalhian" 
school district* Validity data was collected using samples of. 
children's dictation* -A conference held at the Arizona Center this 
Spring, with Dr* Cazden. of Harvard University, has led to a revision 

. of the linguistic variables and iipprovements in the analysis program. 
The success of the CLA-ST has led to , the development of teacher 

, materials to help classroom staff collect -^discrete samples and apply . 
them to children's learning problems ♦ Thfse materials are due to' be 

\:oiiq)leted this, Fall (Conrad, 1975). 
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SESSION I ' 


SESSION II 


SESSION III 


ji!* of Signs FavoJRig TEEM 


1st 


2nd 


3rd 


1st 


2nd 


3rd 


1st 


2nd 


3rd 


1st 


2nd 


3rd 


Total 


LANGUAGE 
PERFORMANCE 


Average if words 
per child 




+ 








+ 




+ 




.3/3 


3/3 


•2/3 


8/9 


Average response 
length 


+ 










^0 








3/3 


2/3 


1/3 


6/9 


LIN(JUTSTIC 
PERiFORMANCE 


Type-Token ratio 


0 


t 

+ 


0 


+ 






+ 


0 


• 


2/3 


1/3 


1 /o 

1/3 


6/9 


Form-Functibn 
ra^iio 


t 

T 


4. 




4. 


4. 




4. 


4. 


n 


J/ J 


J/ J 


1 z*^ 

XZ J 


7 ZQ 

zzy 


Ratio of^pr^s^nt 
tense v^rbs to 
total ve^s^ 


4. 










4. 


4. 


4. 


4. 


Z/ J 


XZ J 


9 Z^ 

zz J 


ZQ 


^ 

Vocabular«v 
diversitjiP 


+ 


+ 




+ 






+ 


+ 


+ 


3/3 


3/3 


2/3 


8/9 


SYNTACTIC 
SRFORMANCE 


\ 

# of T-Units: . 

« 




4- 
1 




4- 


4- 


4- 


0 


4- 


4- 


1 z*^ 
1/ J 


*^ Zq 
JZ J 


Zq 


7 ZO 

zzy 


Type of T-Uhits: 
A 






* 




0 


4- 




0 






X/ J 


1 Z'^ 

X/ J 


^ ZQ 


" B 




4- 


0 






4- 


4- 




4- 




X/ J 


9 Z^ 
z/ J 


zq 




f 

\ 








+ 








+ 


4- 


* 

1/3 




2/3 


5/9 




\ 

"Freq^pcyj of 
child-iniJtiated 
que^^tions 




+ 














+ \ 
'* 1 


2/3 


2/3 


3/3 


7/9 



/ INTERPRETIVE CODE: 
• ^ 4- ' Difference favoring TEEM 

^. - Difference favoring Comparison 
' ;jO No difference 



Table 1. Sign Test Analysis of Language Study Data 

« 
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CONCLUSICN , , . 

♦ * ' > 

The CAOS technique has been Implemented In .a number of school 
districts as a meansXof assessing program implementation tof TEEM. 

- \ 

Teachers have foimd .this ,a .useful way to learn more about the 
effects of the program on the social functioning of children. 

The CLA-rSTl represents a new direction in^he .assessment of 
children's language development at the preschool and early elementary 
level. The development program has led to the conclusion that this 
technique sauries aspects of children's development In a valid 
context,, and produces information for teachers which is applicable 
.to .im^oviijjg .the .instruction of young children. 
. . - Situational .tasks .have proven valuable in the elusive task 
,of -evaluating .an Innovative open classroom p;:ogram.^ Future efforts 
.will be directed toward other areas .of -the -TEEM .program, such as . 
the development of cognitive skills through the Cooking Experience. 
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SELECTED RESOURCES ABOUT TEEM PROGRAM 
AND PROGRAM EVALUATION 

*Arlzona Center for Educatlohal Research and Development. Selected Papers. 
ACBRD, 1974. ^ 

^Arizona <:enter for Educational Research and Development!. Tucson Early 
' Education Model (TEEM) . ACERD, 1974. 

*Goldupp, Ocea. An Investigation of Independent Child .Behavior In the . 
Open Classroom; The Classroom Attitude Observation Schedule (CAPS) > 
October, 1972, ACERD. 

*Grimmett, Sadie. Situational Tests for Evaluation of Intervention Programs : 
A Position Paper , 1970> ACERD mimeo or ERIC ED 049-308. 

. *Rentfrow, Robert K. ; Duming, Kathleen; Conrad, Eva; & Goldupp^ Ocea. Use 
of Three New Instruments in a Head Start Program Evaluation . Reprinted 
from Psychology in the Schools, January, 1975, 12(1), 34-39. ^ 

*Rentfrow, Robert ; & Larson, John C. Correlates of School Achievement in 
Open and Traditional Settings in a Sample of Lpw-Income Children . Paper 
presented at the Biennial Meeting of the Society for Research in Child 
Development Denver , April , 19 75 , , ACERD mimeo. 

*Rentfrdw, Robert^K.; Duming, Kathleen; Conrad, Eva; & -Goldtipp, Oc^a. 

Intensive Evaluation of Head Start Implementation in the Tucs on Early 
Education Model. I972^^ 



In Press : 

*Conrad, Eva; Rentfrow, Robert K. ; Meredith, Keith. Assessment of Languaj^e 
Experien ce Implementation in the Tucson Early Education M^ del. Available 
October 1, 1975. ' I 

*Conrad, Eva. Children's Language Assessment - Situational Tasks: Teacher 
Materials. Available November 1, 1975. 



* All resources ate available from: 

Information Officer 

• ARIZONA CENTER FOR EDUCATIONAL 
RESEARCH AND DEVELOPMENT 
College of Education 
University of Arizona 
Tucson, AZ 85721 
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